In the blink of an eye: Value and novelty drive saccades

Evidence accumulation is an essential component of value-based decisions. Recent human studies suggest that overt attention correlates with evidence accumulation necessary for optimal decisions. However, the influence of covert attention on decision-making remains relatively unexplored. To investigate this issue, two monkeys were trained to perform a decision-making task where they chose between two stimuli, which were either ‘Overtrained’ or learned that day (‘Novel’). Subjects could freely saccade during choice evaluation and indicated their decision by moving a joystick. Saccades were made within 170 ms of stimulus presentation and were strongly driven by both value and novelty, implying covert stimulus evaluation prior to saccade. This effect was strongest for ‘Overtrained’ choices, but rapidly emerged during learning of ‘Novel’ choices. Though novel stimuli attracted initial saccades, final decisions were guided only by value; implying attentional value comparison processes are at least partially dissociable from value comparison processes that govern final decisions. While subjects made highly optimal decisions, they frequently viewed only one stimulus; final choice was thus best explained by assuming covert evidence accumulation. Our results suggest that the primate brain contains multiple value comparison systems for guiding attention toward highly valuable or novel information while simultaneously optimizing final decision value

Mymou: A low-cost, wireless touchscreen system for automated training of non-human primates

Training nonhuman primates (NHPs) to perform cognitive tasks is essential for many neuroscientific investigations, yet laboratory training is a time-consuming process with inherent limitations. Habituating NHPs to the laboratory staff and experimental equipment can take months before NHPs are ready to proceed to the primary tasks. Laboratory training also necessarily separates NHPs from their home-room social group and typically involves some form of restraint or limited mobility, and data collection is often limited to a few hours per day so that multiple NHPs can be trained on the same equipment. Consequently, it can often take a year to train NHPs on complex cognitive tasks. To overcome these issues, we developed a low-cost, open-source, wireless touchscreen training system that can be installed in the home-room environment. The automated device can run continuously all day, including over weekends, without experimenter intervention. The system utilizes real-time facial recognition to initiate subject-specific tasks and provide accurate data logging, without the need for implanted microchips or separation of the NHPs. The system allows NHPs to select their preferred reward on each trial and to work when and for as long as they desire, and it can analyze task performance in real time and adapt the task parameters in order to expedite training. We demonstrate that NHPs consistently use this system on a daily basis to quickly learn complex behavioral tasks. The system therefore addresses many of the welfare and experimental limitations of laboratory-based training of NHPs and provides a platform for wireless electrophysiological investigations in more naturalistic, freely moving environments

Neural Signatures of Value Comparison in Human Cingulate Cortex during Decisions Requiring an Effort-Reward Trade-off

UNLABELLED: Integrating costs and benefits is crucial for optimal decision-making. Although much is known about decisions that involve outcome-related costs (e.g., delay, risk), many of our choices are attached to actions and require an evaluation of the associated motor costs. Yet how the brain incorporates motor costs into choices remains largely unclear. We used human fMRI during choices involving monetary reward and physical effort to identify brain regions that serve as a choice comparator for effort-reward trade-offs. By independently varying both options' effort and reward levels, we were able to identify the neural signature of a comparator mechanism. A network involving supplementary motor area and the caudal portion of dorsal anterior cingulate cortex encoded the difference in reward (positively) and effort levels (negatively) between chosen and unchosen choice options. We next modeled effort-discounted subjective values using a novel behavioral model. This revealed that the same network of regions involving dorsal anterior cingulate cortex and supplementary motor area encoded the difference between the chosen and unchosen options' subjective values, and that activity was best described using a concave model of effort-discounting. In addition, this signal reflected how precisely value determined participants' choices. By contrast, separate signals in supplementary motor area and ventromedial prefrontal cortex correlated with participants' tendency to avoid effort and seek reward, respectively. This suggests that the critical neural signature of decision-making for choices involving motor costs is found in human cingulate cortex and not ventromedial prefrontal cortex as typically reported for outcome-based choice. Furthermore, distinct frontal circuits seem to drive behavior toward reward maximization and effort minimization. SIGNIFICANCE STATEMENT: The neural processes that govern the trade-off between expected benefits and motor costs remain largely unknown. This is striking because energetic requirements play an integral role in our day-to-day choices and instrumental behavior, and a diminished willingness to exert effort is a characteristic feature of a range of neurological disorders. We use a new behavioral characterization of how humans trade off reward maximization with effort minimization to examine the neural signatures that underpin such choices, using BOLD MRI neuroimaging data. We find the critical neural signature of decision-making, a signal that reflects the comparison of value between choice options, in human cingulate cortex, whereas two distinct brain circuits drive behavior toward reward maximization or effort minimization

Capturing the temporal evolution of choice across prefrontal cortex

Activity in prefrontal cortex (PFC) has been richly described using economic models of choice. Yet such descriptions fail to capture the dynamics of decision formation. Describing dynamic neural processes has proven challenging due to the problem of indexing the internal state of PFC and its trial-by-trial variation. Using primate neurophysiology and human magnetoencephalography, we here recover a single-trial index of PFC internal states from multiple simultaneously recorded PFC subregions. This index can explain the origins of neural representations of economic variables in PFC. It describes the relationship between neural dynamics and behaviour in both human and monkey PFC, directly bridging between human neuroimaging data and underlying neuronal activity. Moreover, it reveals a functionally dissociable interaction between orbitofrontal cortex, anterior cingulate cortex and dorsolateral PFC in guiding cost-benefit decisions. We cast our observations in terms of a recurrent neural network model of choice, providing formal links to mechanistic dynamical accounts of decision-making

Transferring structural knowledge across cognitive maps in humans and models

Relations between task elements often follow hidden underlying structural forms such as periodicities or hierarchies, whose inferences fosters performance. However, transferring structural knowledge to novel environments requires flexible representations that are generalizable over particularities of the current environment, such as its stimuli and size. We suggest that humans represent structural forms as abstract basis sets and that in novel tasks, the structural form is inferred and the relevant basis set is transferred. Using a computational model, we show that such representation allows inference of the underlying structural form, important task states, effective behavioural policies and the existence of unobserved state-trajectories. In two experiments, participants learned three abstract graphs during two successive days. We tested how structural knowledge acquired on Day-1 affected Day-2 performance. In line with our model, participants who had a correct structural prior were able to infer the existence of unobserved state-trajectories and appropriate behavioural policies

A circuit mechanism for decision-making biases and NMDA receptor hypofunction

Decision-making biases can be features of normal behaviour, or deficits underlying neuropsychiatric symptoms. We used behavioural psychophysics, spiking-circuit modelling and pharmacological manipulations to explore decision-making biases during evidence integration. Monkeys showed a pro-variance bias (PVB): a preference to choose options with more variable evidence. The PVB was also present in a spiking circuit model, revealing a potential neural mechanism for this behaviour. To model possible effects of NMDA receptor (NMDA-R) antagonism on this behaviour, we simulated the effects of NMDA-R hypofunction onto either excitatory or inhibitory neurons in the model. These were then tested experimentally using the NMDA-R antagonist ketamine, a pharmacological model of schizophrenia. Ketamine yielded an increase in subjects' PVB, consistent with lowered cortical excitation/inhibition balance from NMDA-R hypofunction predominantly onto excitatory neurons. These results provide a circuit-level mechanism that bridges across explanatory scales, from the synaptic to the behavioural, in neuropsychiatric disorders where decision-making biases are prominent

Approach-induced biases in human information sampling

Information sampling is often biased towards seeking evidence that confirms one’s prior beliefs. Despite such biases being a pervasive feature of human behavior, their underlying causes remain unclear. Many accounts of these biases appeal to limitations of human hypothesis testing and cognition, de facto evoking notions of bounded rationality, but neglect more basic aspects of behavioral control. Here we demonstrate involvement of Pavlovian approach biases in determining which information humans will choose to sample. We collected a large novel dataset from 32,445 human subjects, making over 3 million decisions, who played a gambling task designed to measure the latent causes and extent of information-sampling biases. We identified three novel approach-related biases, formalized by comparing subject behavior to a dynamic programming model of optimal information gathering. These biases reflected the amount of information sampled (‘positive evidence approach’), the selection of which information to sample (‘sampling the favorite’), and the interaction between information sampling and subsequent choices (‘rejecting unsampled options’). The prevalence of all three biases was related to a Pavlovian approach-avoid parameter quantified within an entirely independent economic decision task. Our large dataset also revealed that individual differences in information seeking are a stable trait across multiple gameplays, and can be related to demographic measures including age and educational attainment. As well as revealing limitations in cognitive processing, our findings suggest information sampling biases reflect the expression of primitive, yet potentially ecologically adaptive, behavioral repertoires. One such behavior is sampling from options that will eventually be chosen, even when other sources of information are more pertinent for guiding future action

Correction: Approach-Induced Biases in Human Information Sampling

This corrects the title for figure 2 of the article published in vol 14(11), e2000638, https://doi.org/10.1371/journal.pbio.2000638

Reconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex

Competing accounts propose that working memory (WM) is subserved either by persistent activity in single neurons or by dynamic (time-varying) activity across a neural population. Here, we compare these hypotheses across four regions of prefrontal cortex (PFC) in an oculomotor-delayed-response task, where an intervening cue indicated the reward available for a correct saccade. WM representations were strongest in ventrolateral PFC neurons with higher intrinsic temporal stability (time-constant). At the population-level, although a stable mnemonic state was reached during the delay, this tuning geometry was reversed relative to cue-period selectivity, and was disrupted by the reward cue. Single-neuron analysis revealed many neurons switched to coding reward, rather than maintaining task-relevant spatial selectivity until saccade. These results imply WM is fulfilled by dynamic, population-level activity within high time-constant neurons. Rather than persistent activity supporting stable mnemonic representations that bridge subsequent salient stimuli, PFC neurons may stabilise a dynamic population-level process supporting WM

Differential Encoding of Factors Influencing Predicted Reward Value in Monkey Rostral Anterior Cingulate Cortex

Background: The value of a predicted reward can be estimated based on the conjunction of both the intrinsic reward value and the length of time to obtain it. The question we addressed is how the two aspects, reward size and proximity to reward, influence the responses of neurons in rostral anterior cingulate cortex (rACC), a brain region thought to play an important role in reward processing. Methods and Findings: We recorded from single neurons while two monkeys performed a multi-trial reward schedule task. The monkeys performed 1–4 sequential color discrimination trials to obtain a reward of 1–3 liquid drops. There were two task conditions, a valid cue condition, where the number of trials and reward amount were associated with visual cues, and a random cue condition, where the cue was picked from the cue set at random. In the valid cue condition, the neuronal firing is strongly modulated by the predicted reward proximity during the trials. Information about the predicted reward amount is almost absent at those times. In substantial subpopulations, the neuronal responses decreased or increased gradually through schedule progress to the predicted outcome. These two gradually modulating signals could be used to calculate the effect of time on the perception of reward value. In the random cue condition, little information about the reward proximity or reward amount is encoded during the course of the trial before reward delivery, but when the reward is actually delivered the responses reflect both the reward proximity and reward amount